Distributed dynamic programming for discrete-time stochastic control, and idempotent algorithms

Previously, idempotent methods have been found to be extremely fast for solution of dynamic programming equations associated with deterministic control problems. The original methods exploited the idempotent (e.g., max-plus) linearity of the associated semigroup operator. However, it is now known that the curse-of-dimensionality-free idempotent methods do not require this linearity. Instead, it is sufficient that certain solution forms are retained through application of the associated semigroup operator. Here, we see that idempotent methods may be used to solve some classes of stochastic control problems. The key is the use of the idempotent distributive property. This allows one to apply the curse-of-dimensionality-free idempotent approach. We demonstrate this approach for a class of nonlinear, discrete-time stochastic control problems.